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Abstract 


Given  an  acceptable  level  of  military  performance,  arguably  the  two  most 
important  criteria  for  weapon  systems  performance  are  readiness  (operational 
availability,  or  Ao)  and  life  cycle  cost  (LCC).  These  two  criteria  are  in  conflict:  one 
may  maximize  Ao  by  increasing  LCC  or  minimize  LCC  by  degrading  Ao.  In  this 
paper,  we  develop  a  model  to  analyze  risk  factors  associated  with  this  bi-criterion 
problem.  To  analyze  the  impact  of  input  factors  on  risk,  or  variance  in  the  outcome 
criteria,  we  conduct  Monte  Carlo  simulation.  We  then  apply  design  of  experiments 
methodologies  to  identify  risk  factors  by  analyzing  quantiles  of  these  criteria  in  which 
we  examine  the  probability  that  cost  will  exceed  a  certain  threshold  dollar  amount 
and  the  probability  that  readiness  falls  below  a  certain  threshold. 

Keywords:  weapon  systems  performance,  readiness,  life  cycle  cost,  risk 
factors,  Monte  Carlo  simulation 
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I. 


Introduction 


The  life  cycle  cost  (LCC)  of  a  weapon  system  is  the  cost  of  acquiring, 
operating,  and  supporting  that  weapon  system.  The  careful  management  of  LCC 
will  be  increasingly  important  as  defense  budgets  come  under  pressure  in  the  next 
few  years  because  of  the  U.S.  Federal  Government  debt  and  deficit.  There  is  a 
trade-off  between  LCC  and  deployed  performance  of  a  weapon  system  because 
investments  in  support  --  e.g.,  spare  inventory,  repair  capacity,  etc.  are  expensive. 
Since  these  expenditures  are  to  some  degree  discretionary,  program  managers  will 
be  under  increasing  pressure  to  reduce  deployed  support  investments.  The  intent  of 
such  pressure  will  be  to  reduce  waste,  not  to  reduce  performance.  But  program 
managers  need  a  tool  that  can  clearly  show  the  trade-offs  between  cost  and 
performance  in  order  to  demonstrate  the  performance  impact  of  cost-cutting 
measures  and  to  avoid  cutting  more  deeply  into  performance  than  they  intend. 

However,  it  is  difficult  to  accurately  track  the  LCC  of  a  single  weapon  system 
because  so  many  support  facilities  (e.g.,  repair,  training)  are  shared  and  support 
several  weapon  systems,  and  many  costs  are  relatively  fixed.  The  model  we 
present  in  this  paper  tracks  the  LCC  of  multiple  operating  units  of  a  weapon  system 
(e.g.,  multiple  squadrons  of  aircraft).  The  cost  model  incorporates  start-up  (fixed) 
costs  such  as  acquisition  and  maintenance  facilities,  as  well  as  spare  inventory 
pools  for  major  components.  Operations  and  maintenance  costs,  personnel  and 
training  costs,  as  well  as  disposal  costs  are  also  included  in  the  model. 

Investment  and  operating  expenses  are  related  to  operational  availability  (Ao) 
through  a  stochastic  model  that  considers  variability  in  operational  tempo  (Op- 
tempo),  operating  and  maintenance  costs,  component  reliability,  and  maintenance 
downtime.  The  model,  which  relates  costs  to  performance,  incorporates  congestion 
effects  and  the  impact  of  delays  due  to  an  insufficient  investment  in  capacity.  For 
example,  a  smaller  investment  in  spare  parts  inventory  may  lead  to  reductions  in  Ao 
as  weapon  systems  wait  for  depot  repair  of  a  component  part. 
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We  use  our  model  to  analyze  the  trade-offs  between  LCC  and  Ao  across  a 
range  of  input  factors  that  represent  differences  between  (1)  acquisition  program 
approaches,  (2)  depot  maintenance  characteristics,  and  (3)  squadron  operations. 
Some  factors  are  under  the  control  of  decision-makers,  such  as  spare  inventory 
levels,  while  others  are  exogenous,  such  as  the  price  of  petroleum,  oil,  and  lubricant 
(POL).  The  intent  of  including  exogenous  factors  is  to  give  an  idea  of  the  degree  of 
leverage  a  decision-maker  may  have  in  affecting  LCC  or  Ao,  given  the  impact  of 
variables  beyond  his  or  her  control. 

Input  factors  that  are  varied  include  the  mean  time  between  failures  (MTBF) 
of  major  components,  Op-tempo,  POL  cost,  depot  repair  turnaround  time  (DTAT), 
and  preventive  maintenance  (for  depot  overhaul)  turnaround  time  (PMTAT).  Each  of 
these  factors  is  varied  over  several  levels,  creating  an  analysis  problem  too  large  for 
a  full-factorial  experiment.  Instead,  we  used  a  Nearly-Orthogonal  Latin  Hypercube 
(NOLH)  design,  which  allows  a  detailed,  if  not  exhaustive,  analysis  of  the  way  these 
input  variables  impact  the  outcome  criteria. 

In  many  cases,  decision-makers  involved  in  planning  and  operations  for  a 
weapon  system  are  concerned  about  more  than  just  average  LCC  or  average  Ao, 
but  in  addition,  they  want  to  understand  the  impact  of  variability  in  costs  and 
performance.  For  example,  a  decision-maker  may  wish  to  know  the  probability  that 
LCC  exceeds  a  given  budget  (which  we  call  budget  risk  [BR])  or  the  probability  that 
Ao  falls  below  a  certain  planning  threshold  (which  we  call  readiness  risk  [RR];  Kang, 
Doerr,  &  Sanchez,  2006;  Kang,  Doerr,  Apte,  &  Boudreau,  2010).  To  analyze  the 
impact  of  variance  in  the  input  factors  on  risk,  we  conducted  post  hoc  analyses  using 
Monte  Carlo  simulation. 


To  perform  the  post  hoc  analyses,  we  conducted  a  factorial  experiment  on 
three  key  factors,  which  the  main  analysis  indicated  had  an  impact  on  mean 
outcomes  but  which  are  also  under  the  control  of  decision-makers.  Results  of  the 
post  hoc  analyses  showed  the  impact  that  factors  under  the  control  of  decision¬ 
makers  can  have  on  budget  and  readiness  risk,  potentially  even  late  in  the  program. 
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Case  Study 


We  have  developed  a  hypothetical  case  involving  fighter  aircraft,  the  F-XX, 
and  its  acquisition,  operation,  and  maintenance  and  have  created  a  companion 
spreadsheet  model  to  estimate  the  LCC  and  Ao.  We  briefly  describe  the  case  in  the 
next  section.  More  details  of  this  F-XX  case  study  are  described  in  Kang  and  Doerr 
(2012). 

A.  The  Case  and  the  Spreadsheet  Model 

A  total  of  96  new  fighter  aircraft,  F-XX,  will  be  acquired  and  divided  into  eight 
squadrons.  The  F-XX  program  life  cycle  is  estimated  to  be  30  years.  The  manning 
requirements  are  as  follows: 

Pilots  and  Ground  Support  Personnel  (per  squadron) 

■  17  Pilots 

■  4  Ground  Support  Officers 

■  16  Non-commissioned  Officers  (NCOs) 

■  176  Enlisted 

Headquarters  Personnel 

■  2  One  Commanding  Officer  (CO)  and  one  Executive  Officer  (XO) 
(both  are  pilots) 

■  1  Administration  Officer  (non-pilot) 

■  2  NCOs 

■  4  Enlisted 

The  personnel  costs  are  based  on  the  DoD  standard  composite  pay  that 
includes  standard  benefits  (housing,  food,  medical,  etc.,  not  including  re-enlistment 
bonuses,  combat  pay,  etc.).  (See  http://www.defenselink.mil/ 
comptroller/rates/fv2013/  2013  k.pdf.)  All  F-XX  personnel  will  require  both  basic 
and  advanced  levels  of  training.  The  manpower  annual  turnover  rate  is  estimated  to 
be  20%,  and  additional  personnel  must  be  trained  due  to  attrition.  The  cost  for  non¬ 
pilot  basic  training  is  estimated  to  be  $2,000  per  week  per  person,  and  advanced 
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training  is  estimated  to  be  $3,000  per  week  per  person.  Pilot  training  costs  $1 1 ,000 
per  week  per  person  for  both  basic  and  advanced  training.  Required  training  times 
for  pilots  and  ground  maintenance  personnel  and  headquarters  personnel  are  as 
follows: 

Pilots  and  Ground  Personnel  Training  Requirements 


Basic 

Advanced 

Officer 

Pilot 

36  weeks 

1 2  weeks 

Ground 

12  weeks 

2  weeks 

NCO 

1 2  weeks 

2  weeks 

Enlisted 

24  weeks 

24  weeks 

Headquarters  Personnel  Training  Requirements 

Basic  Admin  Training  (Officers,  NCOs,  enlisted):  10  weeks 

Advanced  Admin  Training  (Officers  and  NCOs):  3  weeks 

Management  of  spare  parts  will  be  on  a  one-for-one  exchange  at  the 
squadron  level  (organizational  level,  or  O-level).  It  takes  one  day  to  swap  the  failed 
component  with  a  spare  part,  if  the  part  is  available.  Otherwise,  the  aircraft  will  be 
grounded  until  a  ready-for-issue  (RFI)  spare  part  becomes  available.  An  average 
waiting  time  for  an  RFI  spare  part  is  assumed  to  be  50%  of  the  depot  repair 
turnaround  time.  A  failed  component  is  sent  to  the  contractor-managed  depot  for 
repair.  The  depot  turnaround  time  is  estimated  to  be  40  days.  Each  aircraft  will  go 
through  preventive  maintenance  (overhaul)  every  five  years,  which  takes  an  average 
of  three  months. 

Each  squadron’s  activity  has  start-up  fixed  costs,  which  are  incurred  at 
$10,000,000  per  activity  prior  to  squadron  activation.  Additionally,  operating  variable 
costs,  which  are  estimated  at  $5,000,000  per  year  per  O-level  activity,  are  incurred 
for  each  year  that  a  squadron  is  operational. 
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In  this  study,  we  consider  the  six  major  components  shown  in  Table  1 .  The 
MTBFs  and  the  unit  costs  for  the  components  are  included  in  the  table.  To  make  the 
model  more  realistic,  we  may  need  to  add  some  more  components,  yet  not  too 
many.  As  shown  in  Kang  and  McDonald  (2010)  in  the  case  of  light  armored  vehicles, 
generally  in  any  major  weapon  system,  a  small  number  of  components  contribute  to 
majority  of  the  parts  cost. 

Table  1 .  The  MTBF  and  the  Unit  Cost  of  the  Six  Major  Components 


Component  Name 

MTBF 

Unit  Cost 

Auxiliary  Power  Unit 

250 

$100,000 

Generator 

400 

$  250,000 

Radar 

1000 

$  400,000 

Avionic  Computer 

1000 

$  500,000 

Landing  Gear 

500 

$  400,000 

Engine 

500 

$  2,000,000 

Each  aircraft  is  expected  to  fly  an  average  of  40  hours  per  month,  and  the 
cost  for  POL  is  estimated  to  be  $2,000  per  flying  hour.  Each  copy  of  the  F-XX  has  an 
average  unit  cost  of  $50  million.  Support  equipment  costs  are  $20,000  per  aircraft. 
These  one-time  costs  are  incurred  when  the  F-XXs  are  phased  into  the  squadrons. 
The  expected  salvage  value  of  each  aircraft  at  the  end  of  the  life  cycle  is  estimated 
to  be  $5  million,  which  is  10%  of  the  procurement  unit  cost.  An  annual  capital 
discount  rate  of  2%  is  used  to  compute  the  present  value  of  the  LCC.  (See 
http://www.whitehouse.gov/omb/circulars/a094/a094.html  for  guidance  for  the  capital 
discount  rate.) 

We  have  developed  a  spreadsheet  model  to  compute  the  life  cycle  cost  and 
the  Ao  of  this  F-XX  case.  A  sample  screenshot  of  the  user-interface  page  of  the 
model  is  shown  in  Figure  1. 
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Figure  1.  A  Sample  Screenshot  of  the  F-XX  Life  Cycle  Cost  and 
Readiness  Spreadsheet  User-Interface  Page 


B.  The  Significant  Factors 

We  considered  the  six  major  components  (auxiliary  power  unit  [APU],  engine, 
landing  gear,  avionic  computer,  radar,  and  generator)  shown  in  Table  1.  When  any 
of  these  components  fails,  the  faulty  part  is  removed  from  the  aircraft,  a  spare  is 
installed,  and  the  faulty  part  is  sent  to  the  repair  facility.  After  the  repair  is  complete, 
the  repaired  part  becomes  a  spare  and  is  sent  to  the  spare  pool.  When  a  critical 
part  fails  and  a  spare  is  not  available,  the  aircraft  will  be  non-operational  and 
grounded  until  a  spare  becomes  available.  The  components  listed  in  Table  1,  with 
the  exception  of  the  APU,  are  considered  critical. 


We  have  chosen  the  following  11  factors  to  identify  significant  factors  for  the 
readiness  and  life  cycle  cost  of  the  program: 

■  MTBFs:  six  components’  mean  time  between  failures  (or  reliabilities), 

■  CPL:  critical  protection  level  for  spare  parts  (fill  rate  or  service  level), 

■  FLHRS:  monthly  flight  hours  (operational  tempo), 

■  POL:  cost  of  POL, 

■  DTAT:  depot  corrective  maintenance  turnaround  time,  and 

■  PMTAT:  preventive  maintenance  (or  RESET)  turnaround  time. 
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We  used  an  NOLH  with  257  runs  (Cioppa  &  Lucas,  2007).  This  design  is 
capable  of  handling  up  to  29  factors  without  increasing  the  number  of  scenarios 
(Kleijnen,  Sanchez,  Lucas,  &  Cioppa,  2005;  Sanchez,  2006).  We  have  developed  a 
macro  written  in  Visual  Basic  that  computes  the  life  cycle  cost  and  the  operational 
availability  for  each  scenario. 


We  fit  regression  meta-models  of  the  life  cycle  cost  as  a  function  of  the  1 1 
main  effects,  and  two-way  interaction  of  the  11  input  factors,  using  the  JMP® 
software  package  (SAS  Institute,  2008).  After  noticing  the  impact  on  several  non¬ 
significant  factors,  we  picked  five  significant  factors  for  further  analysis:  CPL,  ENG 
(MTBF  of  engine),  POL,  FLHRS,  and  DTAT.  Then  we  fit  regression  meta-models  of 
the  LCC  as  a  function  of  these  five  main  effects  and  two-way  interaction  of  these  five 
input  factors.  The  results  are  shown  in  Figure  2. 


Sorted  Parameter  Estimates 

) 

Term 

Estimate 

Std  Error 

t  Ratio 

Profc»|t| 

CPL 

183.77269 

0.305283 

601.97 

<.0001* 

FLHRS 

76.425826 

0.198426 

385.16 

J  : 

<.0001* 

POL 

1.3483405 

0.00397 

339.62 

,  |:  : 

<.0001* 

(POL-2500. 02)*(FLHRS-50. 0078) 

0.0270166 

0.000236 

114.60 

J  i  i  i 

<.0001* 

DTAT 

7.7000968 

0.198427 

38.81 

ri  :  :  : 

<.0001* 

ENG 

-0.412583 

0.015881 

-25.98 

[ 

:  :  :  : 

<.0001* 

(FLHRS-50.0078)*(DTAT-50.0078) 

0.1279699 

0.011966 

10.69 

<.0001* 

(ENG-625. 004)*(FLHRS-50. 0078) 

-0.00653 

0.000879 

-7.43 

<.0001* 

(ENG-625. 004)*(DTAT-50. 0078) 

-0.005549 

0.000963 

-5.76 

<.0001* 

(CPL-79.501 9)*(POL-2500. 02) 

0.0007257 

0.000369 

1.97 

0.0502 

(ENG-625. 004)*(POL-2500. 02) 

-2.535e-5 

0.000019 

-1.33 

0.1847 

(POL-2500. 02)*(DTAT-50. 0078) 

0.0003133 

0.000269 

1.17 

0.2444 

(CPL-79.501 9)*(ENG-625.004) 

-0.001446 

0.001547 

-0.93 

0.3508 

(CPL-79.501 9)*(FLHRS-50.0078) 

0.0110574 

0.018175 

0.61 

0.5435 

(CPL-79.501 9)*(DTAT-50.0078) 

-0.007607 

0.018003 

-0.42 

0.6730 

Figure  2.  Sorted  Parameter  Estimates  from  JMP  Analysis  for  Life 

Cycle  Cost 

As  shown  in  Figure  2,  CPL,  FLHRS,  and  the  POL  cost  are  the  major  factors 
that  affect  the  LCC.  Also,  the  two-way  interaction  between  FLHRS  and  POL  has 
significant  impact  on  the  LCC.  It  is  intuitively  correct  since  as  the  POL  cost  and  the 
flight  hours  increase  at  the  same  time;  there  is  an  accelerated  impact  on  the 
increase  of  the  life  cycle  cost.  DTAT  and  ENG,  and  other  two-way  interaction 
factors,  are  not  as  critical  as  far  as  the  LCC  is  concerned. 
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We  also  conducted  the  same  regression  meta-model  analysis  for  Ao.  PMTAT 
is  by  far  the  most  significant  factor,  with  FLHRS  and  DTAT  also  statistically 
significant. 

The  previous  analyses  helped  to  identify  the  critical  factors  for  controlling  the 
LCC  and  Ao  across  the  range  of  significant  inputs  we  varied.  The  bi-criterion 
problem  faced  by  a  decision-maker  is  to  use  the  information  provided  by  a  model 
similar  to  ours  to  reduce  LCC,  without  causing  an  unintentionally  severe  impact  on 
Ao.  Rather  than  looking  across  all  factors  as  they  vary,  the  decision-maker  faces  a 
particular  scenario  and  must  make  decisions  about  whether,  and  how  much,  to  vary 
a  particular  parameter.  To  provide  an  idea  of  how  our  model  might  be  used  to  assist 
in  such  a  decision,  we  took  DTAT,  a  factor  identified  as  important  for  both  LCC  and 
Ao,  and  varied  it  across  its  range  of  likely  values  for  a  particular  scenario  (in  this 
case,  the  mean  values  of  all  other  input  factors).  Results  are  shown  in  Figure  3. 

Note  that  while  Ao  is  linearly  decreasing  in  DTAT,  LCC  is  not  linearly  increasing. 
There  are  levels  of  increase  in  DTAT  that  would  provide  no  significant  cost  reduction 
but  would  degrade  performance. 
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Figure  3.  Depot  Turnaround  Time  Impact  on  Bi-criterion  Decision 
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Risk  Analysis 


Risk  analysis  focuses  not  upon  the  main  outcomes  of  the  expected  LCC  and 
Ao,  but  rather  on  the  potential  variability  in  those  two  metrics.  Variability  is  important 
to  assess  for  at  least  two  reasons.  First,  the  expected  outcome  may  not  be  a 
reliable  predictor.  This  is  especially  true  of  LCC.  Program  managers  may  be  more 
interested  in  the  probability  that  they  will  exceed  their  budget  than  the  fact  that  the 
budget  will  be  sufficient  “on  average.”  Second,  the  average  outcome  may  not  be 
especially  relevant  to  operations.  This  is  especially  true  of  Ao  since  operations  are 
planned  assuming  a  particular  level  of  readiness.  Operational  commanders  may  be 
more  interested  in  knowing  the  probability  that  they  will  be  able  to  execute  a  plan  on 
a  particular  day  rather  than  whether  they  would  be  able  to  execute  the  plan  on 
average. 

There  are  several  potential  risk  metrics  available  to  assess  variability  in  Ao 
and  LCC.  We  have  selected  two  metrics  relevant  to  the  issues  raised  in  the  last 
paragraph.  What  we  call  budget  risk  (BR)  is  the  percentage  chance  that  LCC  will 
exceed  a  given  (cumulative)  dollar  budget.  Readiness  risk  (RR)  is  the  percentage 
chance  that  Ao  will  fail  to  achieve  a  given  availability  target. 

The  analysis  of  main  outcomes  indicated  that  cost — and,  to  a  lesser  extent, 
readiness — are  determined  in  part  by  factors  beyond  the  operational  control  of  a 
program  manager.  POL  and  FLHRS  (or  Op-tempo)  are  both  beyond  his  control,  and 
except  for  expensive  engineering  change  orders,  improvements  to  engine  MBTF  are 
difficult  to  achieve,  especially  after  a  system  is  deployed.  The  risk  analysis  reported 
here  investigated  factors  more  easily  controlled  by  a  program  manager.  We  intended 
to  demonstrate  that  significant  reductions  in  budget  and  operational  risk  can  be 
obtained  by  manipulation  of  factors  directly  under  the  control  of  a  program  manager 
(even  late  in  the  life  cycle  of  a  weapon  system — although  our  cost  figures  always 
refer  to  changes  affecting  the  whole  life  cycle).  In  particular,  we  examined  the  effect 
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of  changes  in  PMTAT,  DTAT,  and  CPL  on  the  risk  of  exceeding  a  $20  billion  budget 
($20b-BR)  and  the  risk  of  failing  to  achieve  an  Ao  of  85%  (85%-RR). 

We  varied  our  three  independent  variables  in  three  levels  in  a  3x3x3  factorial 
design  of  experiments.  At  each  level,  the  independent  variable  was  modeled  by  a 
random  variable.  For  DTAT,  the  three  levels  represented  short,  medium,  and  long 
turnaround  times  and  were  distributed  by  triangular  distributions  of  the  following  (in 
days):  T(10,20,40),  T(20,40,80),  and  T(40,80,160).  All  three  of  these  distributions 
had  a  coefficient  of  variation  (CV)  of  0.27.  For  PMTAT,  the  short,  medium,  and  long 
turnaround  times  were  distributed  by  uniform  distributions  of  the  following  (in  days): 
U(15,  105),  U(22.5,  157.5),  and  U(37.5,  262.5).  All  three  of  these  distributions  had  a 
CV  of  0.43.  For  CPL,  the  three  levels  represented  very  high,  high,  and  moderate  fill 
rates  and  were  distributed  according  to  uniform  distributions  (in  percentage)  of  the 
following:  U(96.6,  99.5),  11(91.25,  98.75),  and  U(80.75,  97.25).  These  distributions 
had  a  CV  of  0.009,  0.023,  and  0.054  (respectively),  meaning  that  higher  protection 
levels  were  also  less  variable. 

Note  that  the  levels  selected  for  the  independent  variables  are  arbitrary  and 
neither  informed  by  historical  data  nor  restricted  carefully  to  provide  a  systematic 
analysis  of  the  relationship  between  the  independent  variables  and  the  outcomes. 
Therefore,  the  implications  that  can  be  drawn  from  our  analysis  are  limited. 

However,  our  goal  was  merely  to  demonstrate  (1)  an  approach  to  risk  analysis,  and 
(2)  the  impact  that  factors  that  can  be  changed  by  a  program  manager  (even  late  in 
a  program)  can  have  on  important  program  risks. 

Marginal  probabilities  are  reported  in  Table  2.  Each  cell  in  this  table  is  the 
average  across  the  3x3=9  conditions  of  the  other  two  factors.  Results  showed  that 
readiness  risk  increases  in  PMTAT  and  DTAT  and  decreases  in  CPL.  Budget  risk 
increases  in  DTAT  and  decreases  in  CPL,  although  only  modestly  (spare  parts  are  a 
small  part  of  the  overall  budget).  Budget  risk  is  relatively  unaffected  by  changes  in 
PMTAT  because  our  model  did  not  include  the  cost  associated  with  the  extra  time  in 
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computing  LCC;  the  main  impact  of  increasing  PMTAT  is  that  the  weapon  system  is 
unavailable  for  longer  periods. 

It  is  worth  noting  that  the  magnitude  of  some  of  the  reported  changes  was 
substantial.  A  9%  decrease  in  CPL  (from  98%  to  89%)  resulted  in  a  23%  decrease 
in  readiness  risk  (from  31%  to  8%).  A  60%  decrease  in  PMTAT  (from  an  average  of 
1 12.5  days  to  an  average  of  45  days)  resulted  in  a  six-fold  decrease  in  readiness 
risk  (from  37%  to  6%).  On  the  other  hand,  only  DTAT  significantly  reduced  budget 
risk.  A  75%  decrease  in  DTAT  (from  an  average  of  80  days  to  an  average  of  20 
days)  reduced  budget  risk  by  half  (from  16%  to  8%). 

Table  2.  Marginal  Probabilities  of  Readiness  Risk  and  Budget  Risk 


Marginal  Probabilities 

PMTAT 

DTAT 

CPL 

85%- 

$20b- 

85%- 

$20b- 

85%- 

$20b- 

RR 

BR 

RR 

BR 

RR 

BR 

Low 

6% 

12% 

9% 

8% 

31% 

10% 

Med 

10% 

12% 

15% 

11% 

14% 

12% 

High 

37% 

11% 

29% 

16% 

8% 

13% 

To  examine  interaction,  we  varied  two  factors  at  a  time  and  examined  the 
response  surface  of  the  marginal  probabilities  (marginal  across  the  third  factor)  of 
each  outcome.  This  gave  us  six  potential  graphs  to  draw.  However,  we  omit  the 
graphs  involving  budget  risk  and  PMTAT  because  PMTAT  has  little  impact  on 
budget  risk.  Figure  4  shows  readiness  risk  as  a  function  of  PMTAT  and  DTAT.  As 
expected  from  the  marginal  results,  readiness  risk  rises  sharply  as  both  turnaround 
times  jointly  increase.  However,  the  function  is  clearly  not  monotonic,  and  there  is  a 
clear  saddle  point  below  the  joint  minimum,  indicating  that  past  a  point,  reductions  in 
turnaround  time  would  yield  diminishing  returns. 


ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY 

NAVAL  POSTGRADUATE  SCHOOL 


-11  - 


Surface  Plot 


Figure  4.  Readiness  Risk  as  a  Function  of  PMTAT  and  DTAT 

Figure  5  shows  readiness  risk  as  a  function  of  DTAT  and  CPL.  Although  less 
complex,  the  pattern  is  the  same  as  that  of  Figure  4:  a  sharp  non-linear  increase  in 
readiness  risk  as  DTAT  and  CPL  are  jointly  degraded  (by  increasing  DTAT  and 
decreasing  CPL),  a  non-monotonic  response  surface,  and  diminishing  marginal 
returns. 

The  graph  of  readiness  risk  as  a  function  of  PMTAT  and  CPL  is  similar  to 
Figures  4  and  5  and  is  omitted.  Instead,  we  present  a  contour  plot  (Figure  6)  that 
clarifies  something  that  the  surface  plots  also  suggest:  that  there  are  multiple  ways 
to  obtain  a  given  level  of  readiness  risk  (as  in  the  surface  charts,  intermediate  points 
are  interpolated).  For  example,  one  can  obtain  readiness  risk  as  low  as  15%,  even 
with  a  PMTAT  as  high  as  80  days,  but  only  with  very  high  (98%)  CPL. 
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Surface  Plot 


CPL 


Figure  5.  Readiness  Risk  as  a  Function  of  CPL  and  DTAT 

Finally,  Figure  7  presents  a  graph  of  budget  risk  as  a  function  of  DTAT  and 
CPL.  Although  it  is  also  non-linear  and  non-monotonic,  it  is  clear  from  the  graph  that 
the  effect  of  changes  to  DTAT  is  far  more  powerful  than  changes  to  CPL  in  reducing 
budget  risk. 
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Contour  Plot  for  RR 
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Figure  6.  Readiness  Risk  as  a  Function  of  CPL  and  PMTAT 
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Figure  7.  Budget  Risk  as  a  Function  of  CPL  and  DTAT 
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IV. 


Summary 


In  summary,  we  first  presented  a  methodology  to  identify  significant  factors 
that  affect  readiness  and  life  cycle  cost  for  a  weapon  system,  using  a  fighter  aircraft 
program  as  an  example.  We  then  demonstrated  a  method  of  examining  the  joint 
impact  of  maintenance  and  sparing  on  readiness  and  budget  risk.  The  impact  is 
interactive,  and  highly  non-linear.  We  have  also  shown  that  while  factors  beyond  the 
control  of  a  program  manager,  such  as  POL  and  FLHRS,  may  be  critical  factors  in 
(average)  Ao  and  life  cycle  cost,  factors  within  the  control  of  a  program  manager 
may  still  affect  a  great  deal  of  the  risk  involved  in  those  metrics. 
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■  Energy  Saving  Contracts/DoD  Mobile  Assets 

■  Financing  DoD  Budget  via  PPPs 

■  Lessons  from  Private  Sector  Capital  Budgeting  for  DoD  Acquisition 
Budgeting  Reform 

■  PPPs  and  Government  Financing 

■  ROI  of  Information  Warfare  Systems 

■  Special  Termination  Liability  in  MDAPs 

■  Strategic  Sourcing 

■  Transaction  Cost  Economics  (TCE)  to  Improve  Cost  Estimates 

Human  Resources 

■  Indefinite  Reenlistment 

■  Individual  Augmentation 

■  Learning  Management  Systems 

■  Moral  Conduct  Waivers  and  First-term  Attrition 

■  Retention 

■  The  Navy’s  Selective  Reenlistment  Bonus  (SRB)  Management  System 

■  Tuition  Assistance 

Logistics  Management 

Analysis  of  LAV  Depot  Maintenance 
Army  LOG  MOD 
ASDS  Product  Support  Analysis 
Cold-chain  Logistics 

Contractors  Supporting  Military  Operations 
Diffusion/Variability  on  Vendor  Performance  Evaluation 
Evolutionary  Acquisition 

Lean  Six  Sigma  to  Reduce  Costs  and  Improve  Readiness 
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■  Naval  Aviation  Maintenance  and  Process  Improvement  (2) 

■  Optimizing  CIWS  Lifecycle  Support  (LCS) 

■  Outsourcing  the  Pearl  Harbor  MK-48  Intermediate  Maintenance 
Activity 

■  Pallet  Management  System 

■  PBL  (4) 

■  Privatization-NOSL/NAWCI 

■  RFID  (6) 

■  Risk  Analysis  for  Performance-based  Logistics 

■  R-TOC  AEGIS  Microwave  Power  Tubes 

■  Sense-and-Respond  Logistics  Network 

■  Strategic  Sourcing 

Program  Management 

■  Building  Collaborative  Capacity 

■  Business  Process  Reengineering  (BPR)  for  LCS  Mission  Module 
Acquisition 

■  Collaborative  IT  Tools  Leveraging  Competence 

■  Contractor  vs.  Organic  Support 

■  Knowledge,  Responsibilities  and  Decision  Rights  in  MDAPs 

■  KVA  Applied  to  AEGIS  and  SSDS 

■  Managing  the  Service  Supply  Chain 

■  Measuring  Uncertainty  in  Earned  Value 

■  Organizational  Modeling  and  Simulation 

■  Public-Private  Partnership 

■  Terminating  Your  Own  Program 

■  Utilizing  Collaborative  and  Three-dimensional  Imaging  Technology 

A  complete  listing  and  electronic  copies  of  published  research  are  available  on  our 
website:  www.acguisitionresearch.net 
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